DOCUMENT RESUME 



ED 376 765 



HE 027 902 



AUTHOR 
TITLE 



PUB DATE 
NOTE 



PUB TYPE 



Kelley, Larry H. 

Utilizing a Graduate Follow-up Survey To Assess 

Ins t itut ional Effect iveness in the Smal 1 Liberal Arts 

Col 1 ege . 

May 94 

23p.; Paper presented at the Meeting of the 
Association for Institutional Research Annual Forum 
(34th, New Orleans, LA, May 1994). For a related 
document, see HE 027 901. 
Reports - Research/Technical (143) — 
Speeches/Conference Papers (150) 



EBP.S PRICE 
DESCRIPTORS 



IDENTIFIERS 



MF01/PC01 Plus Postage. 

^College Graduates; College Outcomes Assessment; 
*Educat i onal Exper i ence ; *Graduat e Surveys ; Higher 
Education; Institutional Evaluation; Private 
Colleges; Satisfaction; ^School Effect ivenes s ; Smal 1 
Colleges; "Student Attitudes; Student Personnel 
Servi ces 

"V i rginia Wes leyan Co 1 1 ege 



ABSTRACT 

This paper reports on a 1992 follow-up survey of 
graduates of Virginia Wesleyan College (VWC) . The target population 
of 1,110 graduates from the classes of 1987 through 1992 were 
surveyed, and 576 responded (51.9 percent). The survey sought 
information on demographics, enrollment history, evaluation of 
cognitive and affective outcomes, overall reactions to college 
experience, evaluation of educational and support services, continued 
education, and employment history after graduation. Responding 
graduates indicated that they were generally satisfied with their 
educational experiences and with the programs and services provided 
by the college. Graduates reported that the school fostered personal 
growth in cognitive and affective areas and that their VWC 
experiences hi>d adequately prepared them for continued educational 
pursuits and/or employment. Areas of concern that emerged from the 
survey included students' experiences in student activities and their 
preparation in computers, responsibilities of citizenship, 
application of scientific principles and methods, and applied 
mathematics. Various college services such as registration process, 
library services, and cost of attending, also received less favorable 
evaluations from the graduates. Includes nine tables. (JB) 
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ABSTRACT 



This paper describes a graduate follow-up survey which was 
recently conducted by a small east coast liberal arts 
college. The purpose of the survey was to determine if 
graduates were satisfied with their educational experiences 
and with the college's programs and services, if they 
perceived personal growth in cognitive and non-cognitive 
areas, and if they felt that the college had adequately 
prepared them for the work force and/ or for continued 
educational pursuits. Results of the survey are presented 
along with a brief overview of follow-up assessment 
activities. The information provided in this paper should be 
useful to other colleges and universities interested in 
assessing educational outcomes and in utilizing survey 
methodologies to do so. 



INTRODUCTION 



In recent years , demands on colleges and universities to 
demonstrate institutional effectiveness have increased 
greatly. In response, a growing number of institutions have 
developed comprehensive assessment plans. Assessment of 
educational outcomes is often a key component of such plans. 

Pace (1985) indicated that students should be more 
knowledgeable after their educational experiences. They 
should improve their ability to write clearly and 
effectively, gain a better understanding of themselves and 
others, develop a greater appreciation of the arts, and 
acquire an understanding of history and social institutions. 

Information provided in Senate Document No. 14 (1986) 
suggested that the college experience also includes changes 
in attitudes, in behavior, in value systems, in the 
inclination toward good citizenship, and in other traits such 
as discipline and self-control. The report went on to say 
that one way to assess achievement in these areas is a survey 
designed to measure the attitudes of students and their level 
of satisfaction with their educational experiences. 

Stevenson, Walleri, and Japely (1985) noted that such a 
survey is a valuable instrument to determine if, after 
sufficient time to reflect, graduates are satisfied with 
the experiences they had in college. 
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Nichols (1991) further indicated that this type of survey can 
provide insight into how graduates have developed' over 
time and valuable information concerning their activities 
since graduation, such as employment experiences and 
continued educational pursuits, 

A small east coast liberal arts college developed a graduate 
follow-up survey which encompasses all of the areas discussed 
above. Through the survey, the college sought to answer the 
following questions: 

1. How do graduates evaluate their educational 
experiences at the college? 

2 • How do graduates evaluate college programs and 
services? 

3 . Did graduates experience personal growth in 
cognitive and non-cognitive areas as students? 

4. Were graduates adequately prepared for employment 
and continued educational pursuits? 

The college conducted its first annual follow-up survey of 
graduates in 1992. The purpose of this paper is to report 
the results of that survey. 



/ / 

VWC GRADUATE FOLLOW-UP SURVEY 

The Office of Institutional Research and Assessment at 
Virginia Wesleyan College (VWC) conducts an annual graduate 
follow-up survey. In 1992, a follow-up survey of students 
who graduated from 1987 through 1992 was conducted. 

Method 

The target population was composed of 1110 graduates. 
The first group of 547 graduated in 1987, 1989, or 1991 and 
the second group of 563 graduated in 1988, 1990, or 1992. 
Initial surveys were sent to the first group in February, 
1992 and to the second group in October of the same year. In 
each case, a second survey was sent approximately one month 
later to those who could not be identified as respondents to 
the initial survey. The survey included questions concerning 
the following areas: 

- Demographics 

- VWC enrollment history 

- Evaluation of cognitive and affective outcomes 

- Overall reactions to college experiences 

- Evaluation of educational and support services 

- Continued educational experiences 

- Employment history after graduation 

A total of 576 completed surveys were returned for an overall 
return rate of 51.9 percent. 
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Results 



A complete description of the results of the 1992 VWC 
graduate follow-up survey have been reported elsewhere 
(Virginia Wesleyan College, Office of Institutional Research 
and Assessment, 1992) . While space does not permit a 
complete description of the findings here, selected results 
will be presented. 

Demographic Profile . The VWC graduate follow-up survey 
includes several items designed to collect general 
demographic information. These data are displayed in 
Table 1. 



Insert Table 1 Here 



Over 65 percent of responding graduates were traditional-age 
students (18 - 23 years of age) . Over 70 percent were female 
and 93 percent were Caucasian-American. 

VWC Enrollment Data . The survey also contains several items 
designed to collect VWC enrollment information. These data 
are illustrated in Tab3 e 2. 



Insert Table 2 Here 



Respondents most often indicated that they decided to attend 
VWC because of its size and location. Over one-half of 
respondents transferred from other colleges or universities. 
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•Nearly three-fourths were commuters at some time during their 
VWC experience and nearly 60 percent commuted the entire 
time. Over 61 percent of responding graduates were enrolled 
at VWC for seven or more semesters. 

Cognitive/Affective Outcomes . The survey asks a series of 
questions regarding students 1 perceptions of the cognitive 
and affective outcomes of their college experiences. A 
majority of respondents reported gains or improvements in 
15 of the 20 areas represented in the survey. Data related 
to areas in which respondents reported substantial growth are 
displayed in Table 3. 



Insert Table 3 Here 



Over 77 percent indicated that their experiences at VWC had 
helped broaden their knowledge of the liberal arts. 
Approximately three-fourths indicated that they had 
experienced personal growth in their abilities to work 
cooperatively in a group and to develop and utilize critical 
thinking skills. Over 72 percent cited improvement in 
written communication skills and in their ability to learn 
actively rather than passively. Over 70 percent reported 
personal growth in their ability to understand different 
philosophies, cultures, and faith perspectives. 

Data related to areas in which respondents indicated low 
levels of positive growth can be seen in Table 4. 
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Insert Table 4 Here 



These included the following: participating in student 
activities, knowing how to use computers, understanding the 
responsibilities of citizenship, understanding and applying 
scientific principles and methods, and understanding and 
applying mathematics in daily activities. 

Overall Reactions to WC . Several survey items deal with 
graduates 1 overall reactions to the college. These data are 
displayed in Table 5. 



Insert Table 5 Here 



Over 91 percent of respondents indicated that apart from the 
financial benefit, VWC had helped improve the quality of 
their life. Nearly 79 percent indicated that they would 
attend VWC again and over 84 percent would recommend it to 
friends. Over 65 percent reported that the quality of 
education provided by VWC was better than that provided by 
other colleges or universities they had attended. 

Evaluation of Educational and Support Services . The survey 
also includes a series of items related to respondents 1 
evaluations of a variety of educational and support services 
at VWC. Data related to items which received very favorable 
ratings from responding graduates are displayed in Table 6. 



Insert Table 6 Here 



Over 97 percent of respondents expressed satisfaction with 
the size of classes. Over 95 percent were satisfied with the 
friendliness of faculty and staff and over 94 percent were 
satisfied with the size of the college, its location, and the 
helpfulness and individual attention of faculty. Over 91 
percent expressed satisfaction with the quality of 
instruction. 

Data related to items receiving less favorable evaluations 
can be seen in Table 7. 



Insert Table 7 Here 



These included the registration process, the availability of 
computers and computer labs, library services, college rules 
and regulations, career planning and placement services, the 
quality of food/service in the dining center, freshman 
seminar, and the cost of attending the college. 

Continued Educational Experiences . Another component of the 
survey deals with graduates' educational experiences after 
VWC. These data are included in Table 8. 



Insert Table 8 Here 



ERLC 



-7- 

10 



Nearly one-third of respondents indicated that they had 
continued their education after graduating from VWC. Over 
two-thirds of these have earned/are pursuing professional or 
graduate degrees. Nearly 97 percent of responding graduates 
indicated that VWC had adequately prepared them for continued 
educational pursuits. 

Employment Since Graduation . The final component of the 
survey deals with respondents 1 employment experiences since 
graduation. These data are illustrated in Table 9. 



Insert Table 9 Here 



Nearly 72 percent of respondents indicated that they are 
either employed full-time, self-employed, or in the military. 
Over' 89 percent indicated that their VWC experiences had 
adequately prepared them for employment. Nearly 63 percent 
reported that their current job is moderately or closely 
related to their major at VWC. 

Discussion 

The results of the graduate follow-up survey reported here 
provide initial answers to the basic questions put forth 
earlier. In general, responding graduates indicated that 
they were satisfied with their educational experiences at VWC 
and with the programs and services provided by the college. 
They also reported personal growth in a variety of cognitive 
and affective areas. In addition, respondents indicated that 



their VWC experiences had adequately prepared them for 
continued educational pursuits and/or employment. Thus, the 
results of the survey provide important evidence that the 
college is doing an effective job in helping students achieve 
key educational outcomes. However, findings also revealed 
several areas of concern which must be addressed if the 
college is to be truly effective. 

The results also confirm that the methodology employed in 
this project can be a valuable component of an overall effort 
to assess an institution's effectiveness. 

Conclusions 

The results of this survey help answer the four major 
questions posed earlier to assess the college's effectiveness 
in providing educational programs and support services which 
enable students to achieve desired educational outcomes. The 
survey demonstrated that, at least at one small liberal arts 
college, graduates are generally satisfied with their 
educational experiences and with college programs and 
services, have experienced personal growth in a variety of 
cognitive and affective areas, and are prepared for 
employment and continued education. 

Although the methodologies employed here provided valuable 
information regarding the overall effectiveness of the 
college, other assessment efforts are needed to further 
investigate those areas of concern revealed by the results. 
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Several such efforts were undertaken recently. First, a 
group of 3 0 faculty and staff members attended a retreat in a 
quiet, secluded setting where they participated in two days 
of intensive discussions about the survey results and other 
issues related to student satisfaction and retention. They 
identified critical concerns and developed plans to address 
them. Consequently, a survey of current students, staff, and 
faculty was conducted to take a closer look at potential 
problem areas and to elicit suggestions for change and/or 
improvement. This survey was followed by focus group 
discussions and other surveys to further pinpoint problem 
areas and to seek solutions. 

The results of this study indicate that the survey described 
in this paper can be an important component of a systematic 
effort to assess institutional effectiveness and may be of 
considerable value as a model for other colleges and 
universities. 
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Table 1 
DEMOGRAPHIC PROFILE 



Number of 

Responses* Percent 



Age Group (n = 568) 

18-23 years of age 372 65,5% 

24 years of age and older 196 34.5 

Gender (n = 564) 

Female 397 70,4% 

Male 167 29.6 

Ethnic Distribution (n = 559) 

Caucasian-American/White 520 93.0% 

Asian/Pacific Islander 16 2.9 

African-American/Black 13 2.3 

Hispanic 3 0.5 

Other/Unknown 7 1.3 



* Calculations for this table were based on the actual number 
responding to each item. 
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Table 2 
VWC ENROLLMENT DATA 



Number of 

Responses* Percent 



Reasons Most Often Given 
for Selecting VWC** 

Size of the college 182 31.6% 

Location of the college 153 26.6 
Availability of scholarships/ 

financial aid 71 12.3 

Academic programs offered 69 12.0 

Academic reputation 68 11.8 

Began Studies at VWC (n = 576) 

Yes 286 49.7% 

No 290 50.3 

Student Residence (n = 576) 

Commuted 344 59.7% 

VWC student housing 147 25.5 

Both 8 5 14.8 

Semesters Enrolled (n = 561) 

2 to 4 86 15.3% 

5 to 6 132 23.5 

7 to 8 286 50.9 

9 to 10 47 8.4 

over 10 10 1.8 



* Calculations for this table were based on the actual 
number responding to each item. 

** Multiple response item; numbers and percentages are not 
additive. 
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Table 3 

PERSONAL GROWTH AREAS RECEIVING POSITIVE EVALUATIONS 



Number of 

Responses Positive Neutral Negative 



ERIC 



Gaining an understanding 
of the liberal arts 572 77.4% i8.4% 4.2% 

Working in a group 566 75.8 18.6 5.7 

Developing critical 
thinking skills 566 74.6 21.2 4.3 

Writing clearly 
& effectively 570 72.4 20.4 7.2 

Learning actively rather 
than passively 563 72.3 20.6 7.1 

Understanding other 
philosophies & cultures 571 70.4 23.3 6.4 



NOTE: Data were collected utilizing a 5-point scale ranging 
from very much to none. Thus, the "positive 11 column 
includes "very much" and "quite a bit" responses 
combined and the "negative" column includes "very 
little" and "none" responses combined. The "neutral" 
column contains "somewhat" responses. 
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Table 4 

PERSONAL GROWTH AREAS RECEIVING LESS FAVORABLE EVALUATIONS 



Number of 

Responses Positive Neutral Negative 



Participating in student 
activities 541 44,9% 32,0% 23,1% 

Understanding how to 
use computers 543 43,1 28,5 28,4 

Learning citizenship 
responsibilities 563 39,5 37,5 23,1 

Utilizing scientific 
principles/methods 549 37,5 38,4 24,0 

Understanding and applying 
mathematics in daily life 554 30.1 34,8 35,1 



NOTE: Data were collected utilizing a 5-point scale ranging 
from very much to none. Thus, the "positive 11 column 
includes "very much" and "quite a bit" responses 
combined and the "negative" column includes "very 
little" and "none" responses combined. The "neutral" 
column contains "somewhat" responses. 
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Table 5 

SATISFACTION WITH WC OVERALL 



Number of 

Responses* Percent 



WC Improved Quality of Life (n=572) 

Definitely 354 61.9% 

Probably s 168 29.4 

Uncertain ' 32 5.6 

Probably not 13 2.3 

Definitely not 5 0.9 

Would Attend VWC Again (n=573) 

Definitely 299 52.2% 

Probably 152 2 6.5 

Uncertain 68 11.9 

Probably not 3 3 5.8 

Definitely not 21 3.7 

Would Recommend VWC to Friends (n=571) 

Yes 482 84.4% 

Unsure 70 12.3 

No 19 3 . 3 

Quality of VWC Education (n=563) 

Much better 145 2 5.8% 

Better 222 39.4 

About the same 106 18.8 

Worse 15 2.7 

Uncertain 75 13.3 



* Calculations for this table were based on the actual number 
responding to each item. 
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Table 6 

ASPECTS OF VWC RECEIVING VERY POSITIVE EVALUATIONS 



Number of 

Responses Satisfied Neutral Dissatisfied 



Size of classes 570 97.2% 2.3% 0.5% 

Faculty friendliness 568 95.3 3.2 1.6 

Faculty helpfulness 569 94.7 4.2 1.1 

College location 566 94.5 4.9 0.5 

Size of college 566 94.2 4.6 1.2 

Quality-instruction 567 91.2 6.9 1.9 



NOTE: Data were collected utilizing a 5-point scale ranging 
from very satisfied to very dissatisfied. Thus, the 
"satisfied" column includes "very satisfied 11 and 
"somewhat satisfied 11 responses combined and the 
"dissatisfied 11 column includes "somewhat dissatisfied" 
and "very dissatisfied" responses combined. 
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Table 7 

ASPECTS OF VWC RECEIVING LESS FAVORABLE EVALUATIONS 
Number of 

Responses Satisfied Neutral Dissatisfied 



Registration process 567 57.7% 21.0% 21.3% 

Computers/labs 498 52.2 26.7 21.1 

Library Services 564 51.8 24.6 23.6 

Rules/regulations 542 43.2 37.5 19.4 

Career planning 506 37.5 33.6 28.9 

Dining Center 469 37.5 41.8 20.7 

Freshman seminar 400 31.5 43.3 25.3 

Cost of attending 559 25.8 31.1 43.1 



NOTE: Data were collected utilizing a 5-point scale ranging 
from very satisfied to very dissatisfied. Thus, the 
"satisfied" column includes "very satisfied" and 
"somewhat satisfied" responses combined and the 
"dissatisfied" column includes "somewhat dissatisfied" 
and "very dissatisfied" responses combined. 
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Table 8 

CONTINUED EDUCATIONAL EXPERIENCES 



Number of 

Responses Percent 



Attended College after VWC * 



Yes 


190 


33 


. 0% 


No/Unknown 


386 


67 


. 0 


Reason Continued Education 








To obtain a professional/graduate degree 


127 


66 


.8% 


To obtain/maintain certification/license 


36 


18 


.9 


To improve job skills or knowledge 


13 


6 


.8 


To take courses for self- improvement 


6 


3 


.2 


Other/Unknown 


8 


4 


.2 


VWC Preparation for Continued Education 








Excellent 


85 


44 


.7% 


More than adequate 


57 


30 


.0 


Adequate 


42 


22 


.1 


Less than adequate 


3 


1 


.6 


Unknown 


3 


1 


.6 



* 190 yes responses to this item were the basis for other 
calculations in this table. 



Table 9 
EMPLOYMENT PROFILE 



Number of 

Responses* Percent 



Current Employment Status (n = 557) 

Employed full-time 375 67,3% 

Employed part-time 58 10,4 

Self-employed 13 2 • 3 

Military 12 2,2 

Unemployed 48 8,6 

Other 51 9,2 

VWC Preparation for Employment (n = 441) 

Excellent 102 23 . 1% 

More than adequate 146 33*1 

Adequate 145 32*9 

Less than adequate 21 4.8 

Poor 27 6.1 

Relation of Current Job to Major (n = 455) 

Closely related 204 44,8% 

Moderately related 82 18.0 

Slightly related 65 14.3 

Not related 104 22.9 



* Calculations for this table were based on the actual number 
responding to each item. 



